Until recently, nearly all television receivers were manufactured having a 4:3 aspect ratio; i.e., the ratio of the width of the television screen to the height of the television screen is approximately 4:3. Television receivers have this 4:3 aspect ratio because until recently existing television broadcast standards, such as NTSC (National Television Standards Committee) and PAL (Phase Alternating Line), provided for the production and broadcast of television programming in the 4:3 aspect ratio.
In recent years, television manufacturers have began producing television receivers, video projectors, and flat panel displays having a wider 16:9 aspect ratio in anticipation of new television broadcast standards. Currently, only a small amount of television programming is produced in the 16:9 aspect ratio, although television programming is expected to eventually be produced entirely (or at least substantially) in the 16:9 aspect ratio. The new digital television standard (HDTV—High-Definition Television) will typically capture and store television program content using a 16:9 aspect ratio instead of the 4:3 aspect ratio supported by millions of existing television receivers. To avoid obsoleting 4:3 aspect ratio televisions, systems will be required to convert or modify HDTV signals to allow reception and display of the new signals by televisions having a 4:3 aspect ratio.
FIG. 1 illustrates a conventional digital broadcast system for producing and distributing television programs. The programming content is captured using a video camera 10 that transfers the captured images onto video tape or another storage medium. Alternatively, programming content may be captured using a film telecine, a device that scans and digitizes film images. Later, the captured images are edited into one or more television programs using a video editor 12. A video compressor 14 compresses the television program content to reduce the transmission bandwidth and/or transmission speed required to transmit the television program. Various compression techniques may be used to compress the television program, such as the MPEG-2 (Moving Picture Experts Group) compression algorithm.
The compressed television program is provided to a transmitter 16, which transmits the program to multiple receivers 18 across a communication link 20. Communication link 20 may be, for example, a physical cable, a satellite link, a terrestrial broadcast, an Internet connection, a physical medium (such as a digital video disc (DVD)) or a combination thereof. A decompressor 22 decompresses the signal received by receiver 18 using the appropriate decompression technique. The decompressed television program signal is then displayed on television display 24. Receiver 18 may be a separate component (such as a cable television box) or may be integrated into television display 24. Similarly, decompressor 22 may be a separate component or may be integrated into the receiver or the television display.
FIG. 2 illustrates a conventional layered encoding system that separates a high-resolution image (e.g., an image captured by video camera 10 in FIG. 1 or a film telecine) into multiple layers. This layered encoding system can be used to “bridge” the transition from older low-resolution 4:3 aspect ratio televisions to the newer high-resolution 16:9 aspect ratio televisions. Rather than requiring replacement of over 300 million 4:3 aspect ratio televisions, the layered encoding system separates the high-resolution image into a low-resolution layer (referred to as the “base layer”) and a high-resolution layer (referred to as the “enhancement layer”). The base layer is used by existing low-resolution 4:3 aspect ratio televisions that cannot utilize the higher resolution portions of the image contained in the enhancement layer. Thus, 4:3 aspect ratio televisions can continue to display television programs distributed in the new high-resolution format. High-resolution televisions, such as HDTVs, use both the base layer and the enhancement layer to generate a high-resolution image on the television.
In FIG. 2, a high-resolution source image 30 is separated into a base layer 32 and an enhancement layer 34. The base layer 32 contains information used by all television systems (i.e., both high-resolution and low-resolution television systems). Images produced from the base layer 32 alone are typically comparable in quality to existing low-resolution television images distributed to low-resolution television systems today. The enhancement layer 34 enhances the base layer 32 to provide a high-resolution image. Thus, the enhancement layer 34 is not used by low-resolution television systems.
The enhancement layer 34 is compressed by a compressor 36 and the base layer 32 is compressed by a compressor 38. Each compressor 36 and 38 communicates its compressed data to a transmitter 40 for transmission to one or more television receivers. In the example of FIG. 2, the high-resolution source image 30 is captured by the video camera in a 4:3 aspect ratio. Both the base layer 32 and the enhancement layer 34 also have a 4:3 aspect ratio. Since the source image has a 4:3 aspect ratio, the procedures and/or components that separate out the base layer and the enhancement layer inherently maintain the same aspect ratio for the base layer and the enhancement layer.
The 4:3 aspect ratio base layer 32 is easily processed by a low-resolution television system, because the television system has the same aspect ratio. However, when the 4:3 aspect ratio layers (base layer 32 and enhancement layer 34) are processed by a 16:9 aspect ratio high-resolution display system, the 4:3 image does not properly fit on the 16:9 television screen. Although most high-resolution televisions are being manufactured in the 16:9 aspect ratio, electronic imagery continues to capture images in a 4:3 aspect ratio. Thus, the source image aspect ratio may differ from the high-resolution display aspect ratio.
FIGS. 3A and 3B illustrate two possible methods for displaying an image having a 4:3 aspect ratio on a display having a 16:9 aspect ratio. In FIG. 3A, a 16:9 television screen 100 is not completely filled by the 4:3 image (located between the broken lines). Thus, blank side bars 102 are located on opposite sides of the screen 100. Although the entire 4:3 image is displayed on the 16:9 screen, irregular screen usage results from this arrangement. The two side bars 102 are typically black because that portion of screen 100 is not activated. Therefore, the screen is not utilized in a uniform manner. This irregular activation of the screen may cause a television with a tube screen to “wear” unevenly. For example, the phosphor coating in the middle portion of the screen is regularly activated by the 4:3 image, but the phosphor coating on the two side bars 102 is not activated. When a 16:9 image is displayed on the “worn” screen, the two side bars will be noticeable due to different brightness or contrast between the two side bars and the middle portion of the screen. This type of irregularity permanently diminishes the user's viewing experience.
Another alternative for displaying a 4:3 image on a 16:9 screen is shown in FIG. 3B. In this situation, the width of the 4:3 image is expanded to align with the sides of the 16:9 screen 100. However, this expansion causes the top and bottom portions of the image to extend past the physical limits of the screen 100. Thus, top and bottom portions 110 of the image are not displayed on the screen 100. This arrangement is problematic because portions of the image that were intended to be seen are not visible. For example, the tops of actors' heads may be deleted, resulting in an unnatural appearance. Additionally, any footer information displayed along the bottom of the 4:3 image (e.g., stock tickers, sports scores, etc.) may not be displayed on the screen 100. Thus, neither method for displaying a 4:3 image on a 16:9 display provides a high-quality, problem-free image.
As more 16:9 high-resolution televisions are installed, video cameras are likely to begin transitioning to capture images in the same 16:9 aspect ratio. In this situation, the source image aspect ratio matches the display aspect ratio, thereby eliminating the problems discussed above with respect to FIGS. 3A and 3B. However, the use of 16:9 video cameras creates another problem: the 16:9 image does not properly fit in the existing low-resolution televisions having a 4:3 aspect ratio. When using a 16:9 video camera, the source image has a 16:9 aspect ratio and both the base layer and the enhancement layer also have a 16:9 aspect ratio.
FIGS. 4A, 4B and 4C illustrate methods for displaying an image having a 16:9 aspect ratio on a display having a 4:3 aspect ratio. In FIG. 4A, a television screen 120 has a 4:3 aspect ratio. The 16:9 image 122 is positioned between the two broken lines. Since the aspect ratio of the image is different from the aspect ratio of the screen 120, two blank bars 124 are created across the top and bottom of the screen. These two blank bars 124 may cause screen “wear” problems similar to those caused by the two side bars 102 discussed above with respect to FIG. 3A. The display format shown in FIG. 4A is commonly referred to as “letterboxing.”
In FIG. 4B, the height of the 16:9 image is expanded to align with the top and bottom edges of the 4:3 screen 120. However, this expansion causes the sides of the image to extend past the physical limits of the screen 120. Thus, side portions 130 of the image are not displayed on the screen 120. This arrangement is undesirable because portions of the image that the creator (or editor) intended to be seen by the viewer are not visible. The display format shown in FIG. 4B is commonly referred to as “overscanning.”
In FIG. 4C, a 16:9 image is fit onto a 4:3 aspect ratio screen by using an “anamorphic squeeze” procedure. The anamorphic squeeze procedure laterally (i.e., horizontally) compresses the image, but maintains the top and bottom edges of the image aligned with the 4:3 screen. This lateral compression of the 16:9 image causes some distortion because the image is compressed horizontally, but is not compressed vertically. In the example of FIG. 4C, the original 16:9 image is wider than the 4:3 screen 120, leaving side portions 130 that are not displayed. The anamorphic squeeze procedure compresses the 16:9 image horizontally until the two sides of the 16:9 image align with the sides of screen 120, thereby eliminating side portions 130. A resulting anamorphically squeezed image 132 is displayed on screen 120 without deleting any portion of the image. However, as mentioned above, the image is distorted due to the horizontal compression without any corresponding vertical compression.
Thus, the common aspect ratio for both the base layer and the enhancement layer presents display problems because the aspect ratio of at least one of the layers will not match the aspect ratio of the television on which the layer is to be displayed. The present invention addresses these disadvantages, providing improved coding and decoding of layered image data.